Multi-modal Sensor Fusion for Learning Rich Models for Interacting Soft Robots

Soft robots are typically approximated as low-dimensional systems, especially when learning-based methods are used. This leads to models that are limited in their capability to predict the large number of deformation modes and interactions that a soft robot can have. In this work, we present a deep-learning methodology to learn high-dimensional visual models of a soft robot combining multimodal sensorimotor information. The models are learned in an end-to-end fashion, thereby requiring no intermediate sensor processing or grounding of data. The capabilities and advantages of such a modelling approach are shown on a soft anthropomorphic finger with embedded soft sensors. We also show that how such an approach can be extended to develop higher level cognitive functions like identification of the self and the external environment and acquiring object manipulation skills. This work is a step towards the integration of soft robotics and developmental robotics architectures to create the next generation of intelligent soft robots

Joint Entropy-Based Morphology Optimization of Soft Strain Sensor Networks for Functional Robustness

Dense and distributed tactile sensors are critical for robots to achieve human-like manipulation skills. Soft robotic sensors are a potential technological solution to obtain the required high dimensional sensory information unobtrusively. However, the design of this new class of sensors is still based on human intuition or derived from traditional flex sensors. This work is a first step towards automated design of soft sensor morphologies based on optimization of information theory metrics and machine learning. Elementary simulation models are used to develop the optimized sensor morphologies that are more accurate and robust with the same number of sensors. Same configurations are replicated experimentally to validate the feasibility of such an approach for practical applications. Furthermore, we present a novel technique for drift compensation in soft strain sensors that allows us to obtain accurate contact localization. This work is an effort towards transferring the paradigm of \textit {morphological computation} from soft actuator designing to soft sensor designing for high performance, resilient tactile sensory networks.uture and Emerging Technologies (FET) programme of the European Commission (grant agreement ID 828818)

Drift-Free Latent Space Representation for Soft Strain Sensors

Soft strain sensors are becoming increasingly popular for obtaining tactile information in soft robotic applications. Diverse technological solutions are being investigated to design these sensors. Simultaneously, new methods for modeling these sensor are being proposed due to their highly nonlinear, time varying properties. Among them, machine learning based approaches, particularly using dynamic recurrent neural networks look the most promising. However, these complex networks have large number of free parameters to be tuned, making it difficult to apply them for real-world applications. This paper introduces the concept of transfer learning for modelling soft strain sensors, which allows us to utilize information learned in one task to be applied to another task. We demonstrate this technique on a passive anthropomorphic finger with embedded strain sensors used for two regression tasks. We show how the transfer learning approach can drastically reduce the number of free parameters to be tuned for learning new skills. This work is an important step towards scaling of sensor networks (algorithm-wise) and for using soft sensor data for high-level control tasks

Automated Fruit Quality Testing using an Electrical Impedance Tomography-Enabled Soft Robotic Gripper

Soft robotic grippers are becoming increasingly popular for agricultural and logistics automation. Their passive conformability enables them to adapt to varying product shapes and sizes, providing stable large-area grasps. This work presents a novel methodology for combining soft robotic grippers with electrical impedance tomography-based sensors to infer intrinsic properties of grasped fruits. We use a Fin Ray soft robotic finger with embedded microspines to grab and obtain rich multi-direction electrical properties of the object. Learning-based techniques are then used to infer the desired fruit properties. The framework is extensively tested and validated on multiple fruit groups. Our results show that ripeness parameters and even weight of the grasped fruit can be estimated with reasonable accuracy autonomously using the proposed system

Design and Development of a Hydrogel-based Soft Sensor for Multi-Axis Force Control

As soft robotic systems become increasingly complex, there is a need to develop sensory systems which can provide rich state information to the robot for feedback control. Multi-axis force sensing and control is one of the less explored problems in this domain. There are numerous challenges in the development of a multi-axis soft sensor: from the design and fabrication to the data processing and modelling. This work presents the design and development of a novel multi-axis soft sensor using a gelatin-based ionic hydrogel and 3D printing technology. A learning-based modelling approach coupled with sensor redundancy is developed to model the environmentally dependent soft sensors. Numerous real-time experiments are conducted to test the performance of the sensor and its applicability in closed-loop control tasks at 20 Hz. Our results indicate that the soft sensor can predict force values and orientation angle within 4% and 7% of their total range, respectively

Sensorized Skin With Biomimetic Tactility Features Based on Artificial Cross-Talk of Bimodal Resistive Sensory Inputs

Tactility in biological organisms is a faculty that relies on a variety of specialized receptors. The bimodal sensorized skin, featured in this study, combines soft resistive composites that attribute the skin with mechano- and thermoreceptive capabilities. Mimicking the position of the different natural receptors in different depths of the skin layers, a multi-layer arrangement of the soft resistive composites is achieved. However, the magnitude of the signal response and the localization ability of the stimulus change with lighter presses of the bimodal skin. Hence, a learning-based approach is employed that can help achieve predictions about the stimulus using 4500 probes. Similar to the cognitive functions in the human brain, the cross-talk of sensory information between the two types of sensory information allows the learning architecture to make more accurate predictions of localization, depth, and temperature of the stimulus contiguously. Localization accuracies of 1.8 mm, depth errors of 0.22 mm, and temperature errors of 8.2 °C using 8 mechanoreceptive and 8 thermoreceptive sensing elements are achieved for the smaller inter-element distances. Combining the bimodal sensing multilayer skins with the neural network learning approach brings the artificial tactile interface one step closer to imitating the sensory capabilities of biological skin

3D Printable Sensorized Soft Gelatin Hydrogel for Multi-Material Soft Structures

The ability to 3D print soft materials with integrated strain sensors enables significant flexibility for the design and fabrication of soft robots. Hydrogels provide an interesting alternative to traditional soft robot materials, allowing for more varied fabrication techniques. In this work, we investigate the 3D printing of a gelatin-glycerol hydrogel, where transglutaminase is used to catalyse the crosslinking of the hydrogel such that its material properties can be controlled for 3D printing. By including electron-conductive elements (aqueous carbon black) in the hydrogel we can create highly flexible and linear soft strain sensors. We present a first investigation into adapting a desktop 3D printer and optimizing its control parameters to fabricate sensorized 2D and 3D structures which can undergo >300% strain and show a response to strain which is highly linear and synchronous. To demonstrate the capabilities of this material and fabrication approach, we produce some example 2D and 3D structures and show their sensing capabilities

Using Redundant and Disjoint Time-Variant Soft Robotic Sensors for Accurate Static State Estimation

Soft robotic sensors have been limited in their applications due to their highly nonlinear time variant behavior. Current studies are either looking into techniques to improve the mechano-electrical properties of these sensors or into modelling algorithms that account for the history of each sensor. Here, we present a method for combining multi-material soft strain sensors to obtain equivalent higher quality sensors; better than each of the individual strain sensors. The core idea behind this work is to use a combination of redundant and disjoint strain sensors to compensate for the time-variant hidden states of a soft-bodied system, to finally obtain the true strain state in a static manner using a learning-based approach. We provide methods to develop these variable sensors and metrics to estimate their dissimilarity and efficacy of each sensor combinations, which can double down as a benchmarking tool for soft robotic sensors. The proposed approach is experimentally validated on a pneumatic actuator with embedded soft strain sensors. Our results show that static data from a combination of nonlinear time variant strain sensors is sufficient to accurately estimate the strain state of a system.Future and Emerging Technologies (FET) programme of the European Commission (grant agreement ID 828818

Towards Growing Robots: A Piecewise Morphology-Controller Co-adaptation Strategy for Legged Locomotion

Control of robots has largely been based on the assumption of a fixed morphology. Accordingly, robot designs have been stationary in time, except for the case of modular robots. Any drastic change in morphology, hence, requires a remodelling of the controller. This work takes inspiration from developmental robotics to present a piecewise morphology-controller growth/adaptation strategy that facilitates fast and reliable control adaptation to growing robots. We demonstrate our methodology on a simple 3 degree of freedom walking robot with adjustable foot lengths and with varying inertial conditions. Our results show not only the effectiveness and reliability of the piecewise morphology controller co-adaptation (PMCCA) strategy, but also highlight the need for morphological adaptation as a robot design strategy